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Abstract 

Australian higher education lags behind other western countries in relation to access and attainment of 
equity groups (Organisation for Economic Development [OECD], 2010). This is despite major policy 
reforms in the last two decades targeting better educational outcomes for students from disadvantaged 
backgrounds. The Australian Labor Government’s Higher Education Participation and Partnerships 
Program (HEPPP) creates a new demand for evaluation methods that can establish whether 
interventions funded under the HEPPP contribute to the Commonwealth's policy goal of increasing 
the participation of low socioeconomic status students in university to 20% by 2020 (Oriel, 2011a). 

In 2011, AAIR received funding from the Australian Learning and Teaching Council (ALTC) to 
conduct a study to track student success. AAIR is undertaking the study in conjunction with Deakin 
University and Southern Cross University. Both universities have HEPPP funds available to undertake 
student cohort tracking programs. The initial phase of the ALTC study is focusing on how each of the 
participating universities approach their cohort tracking projects, the systems they are developing for 
capturing the essential data, and the lessons learnt as they progress their projects. The study is also 
considering comparative case studies both in Australia and internationally especially as they concern 
disadvantaged students. 

To better understand how to track these student cohorts, what attributes we need to focus on, over 
what period do we need to track them and why we are tracking them, this study has adopted an action 
learning methodology. Action Learning Teams (ALTs) have been established at Deakin University 
and at Southern Cross University. 

During this study, the project team will be monitoring its own learning pathways, activities and 
outcomes. Lessons learnt will contribute to the knowledge other institutions can take into their own 
cohort tracking projects. This article discusses the questions posed and challenges raised in the initial 
phases of conducting cohort tracking at each university. 
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Background and rationale 

Inequalities in educational attainment remain despite the advancements of policy 
frameworks such as A Fair Chance for All in the 1990s and, more recently, the Review of 
Higher Education. This is especially true for people of low socioeconomic status (SES), 
people with disabilities and women—especially in non-traditional fields of study and higher 
degrees, rural and remote backgrounds and Indigenous people (James, 2005; James, Bexley, 
& Maxwell, 2008). Inequities in access, participation, success, retention and completion still 
occur for those programs with the greatest demand, and first-year attrition is a significant 
issue, especially for Indigenous and remote students. 

The Bradley Review (Department of Education and Workplace Relations, 2008) linked 
expansion of the higher education sector to achieving equity targets, a policy move that is 
mirrored in widening participation policies in other countries (such as the UK). Bradley 
suggested that as well as income support, the criteria required for expansion were new 
institutions, expansion of the academic workforce, institutional mission differentiation, better 
access via open entry, curriculum innovation for accessibility and relevance, and strategies 
for improving retention in the first year (James, 2009). 

University structures and policies, however, are geared towards perpetuating a social 
order that is mirrored in the K12 sector. It is therefore not surprising that James (2005) 
suggests that it is difficult to understand the precise contribution that equity policies and 
programs have had on targets achieved to date. Isolated and remote students and Indigenous 
students are still participating at a rate that is less than half their incidence in the broader 
population (Martin, 2005). If you exclude Indigenous and remote students, retention rates of 
equity groups are on par with other students generally. Even so, attrition rates in many 
universities are much higher than could be considered acceptable. This does not instil 
confidence that the sector can deliver successful outcomes to students who are from the most 
disadvantaged groups in Australian society. In fact, the opposite may be true. 

Part of the problem is that narratives around ‘student choices’ and ‘diversity’ have 
been poorly aligned with equity policy (James, 2005). There has been a narrow equating of 
socioeconomic disadvantage with financial hardship and a focus by educators on concepts of 
external barriers to access and success. Instead, researchers such as Ramsay (1999), James 
(2005) and Marginson (2009) have suggested that universities should focus on the causes of 
disadvantage (e.g., those which reside in families, communities and schools) and should also 
focus on boosting the ‘encouraging factors’ that enable access and attainment for equity 
groups in higher education. 

This article describes the first phase of a longitudinal study to monitor outcomes 
related to student cohorts more effectively at the institutional level in order to gain a better 
understanding of the root causes of disadvantage, as well as tracking the enablers of access, 
retention and success for students from diverse backgrounds. Tracking students in their day- 
to-day activities is an empirical means by which we can identify those factors that hinder 
progress. In this project we will use a tracking system that follows a cohort of students who 
come from diverse backgrounds in order to identify those factors in their life that impede or 
‘boost’ academic excellence. A process for monitoring and reporting on the educational 
progress of cohorts of students (cohort tracking system) would enable a large-scale impact 
assessment of current learning and teaching activities employed to improve underprepared 
students’ success and completion in higher education, such as inclusive pedagogy, bridging 
courses, mentoring and scholarships. Student cohort tracking enables the scientific evaluation 
of university participation activities under the Higher Education Participation and 
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Partnerships Program and complements evaluations of university-school partnerships being 
designed in Victoria under the Commonwealth guidelines by monitoring students across the 
educational life-cycle from point of contact in school to degree completion. 

The higher education sector as a whole has grappled with the development of student 
cohort systems that bring together people, processes and technologies. Universities are rich in 
data. However, data warehousing and business intelligence systems are often naive and 
unsophisticated. This results in limited access to the range of institutional data required to 
identify and track student cohorts and reduces the capacity to analyse such data and apply 
predictive analytics to ‘at risk’ target groups. 

Approach 

The study involves conducting comparative case studies across two universities on the 
people (including partnerships), processes and technologies required to track student cohorts. 
Partner sites include Deakin University with campuses in Victoria and Southern Cross 
University (SCU) based in northern New South Wales. 

This study will deploy an action learning methodology (Patton, 1997) with teams within two 
institutions who will design and implement student cohort tracking (over 5 years) as they 
learn from international and shared practice, and document their own learning. Action 
learning is defined as a strategy by which people learn with and from each other as they 
attempt to identify and then implement solutions to their problems or developmental issues. 
This is achieved through evaluation cycles that involve planning; action, observation and then 
reflection (see Figure 1). ‘Action research takes its cues—its questions, puzzles and 
problems—from the perceptions of practitioners within particular, local practice contexts.’ 
(Argyris & Schon, 1991 cited in Dickens & Watkins, 2006, p. 187). There are three essential 
features that must be present for an activity to be legitimately an action learning program. 
These are: 

• There must be action in the real world rather than in some simulation; 

• The activity must be conducted in a way which involves others, especially other 
participants who are working on the same or quite different projects; and 

• The emphasis must be upon learning; not just the taking of action and this is what 
distinguishes action learning from project team membership. (Patton, 1997) 
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Figure 1. Kurt Lewin’s Spiral Model, Action Research Cycle. 

This study aims to measure the impact of teaching and learning activities to improve 
the participation, progress and completion of students from low SES, low income and 
academically underprepared backgrounds. 

Action learning teams: Process 

The first stage of the study involves designing the appropriate elements necessary to 
advance the tracking of student cohorts. ALTs are: 

• Considering criteria that enable and/or inhibit the sustainability of student tracking over 
time 

• Learning through international and shared practice 

• Learning through research. 

These learning processes are inter-dependent and occur at various stages of the study, 
not necessarily in a linear sequence. 

Considering criteria for sustainability of student tracking over time 

ALTs are collecting data about their own learning process as they embark on the 
design of their student cohort systems. This involves considering the following conditions 
required to enable the sustainability of student tracking over time (note that ALTs are also 
considering other questions and criteria during the conduct of the study): 

People: 

• Who needs to be involved? 
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• What governance is required? 

• Which stakeholders need to be included and how? 

• Do we need partnerships? Which ones? 

• How do all these people need to behave towards one another? 

• Which relationships are in place now? Which need to be established? 

Processes: 

• What processes are involved? 

• What information do teaching staff require that relate to cohort characteristics? What do 
they already get? 

• Are there processes in place to intervene with students ‘at risk’ related to cohort 
characteristics? 

• What teaching and learning interventions are currently in place? 

• Should any of the current processes be changed? Should new processes be established? 
Technologies: 

• Are current student administration systems enabling of student cohort tracking and if not, 
where are modifications required? 

• Are current data warehousing systems enabling of student cohort tracking and reporting 
and if not, where are modifications required? 

• Does one technology allow both the storage and reporting of cohort data? 

• Are there new/emerging technologies that can assist in the tracking of student cohorts? 

• Collaborative technologies; what are they? Are they needed to track student cohorts? 

Learning through international and shared practice 

ALTs are facilitated by the Project Leader and Research Fellow in the documentation 
of their learning process. The Project Team are providing ALTs with expert advice through a 
literature review on the features of effective student cohort tracking systems internationally. 

Teams from each institution are planning to share their learning with the other 
institution, including discussing the criteria that enable or inhibit the design of effective 
student cohort tracking. 

Learning through research 

ALTs are collecting a number of baseline measures, such as perceptions of teaching 
staff and an audit of current processes. They are participating in key informant interviews to 
enable the documentation of the learning process at the end of the first stage of the project. In 
total, around 30 key informant interviews will be conducted early in 2012 comprising ALT 
members, stakeholders related to partnerships with schools, and any process leaders that are 
identified through the audit of current processes. This data will form baseline measures so 
any changes over time in staff perceptions and practices as a consequence of conducting this 
research can be monitored. 

The Project Team is also monitoring what it is learning from its own observations of 
the two ALTs, its processes for observing them, how it has structured its methodology and 
how the team adapts to the things it leams from its observations. As Deakin and SCU’s 
approaches differ, the Project Team needs to be mindful of these differences and to respond 
accordingly. This inevitably adds another level of complexity to the process and additional 
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burden on the support the research team can provide. In many ways, the addition of the 
ALTC Project on top of the already high demands imposed by the HEPPP projects is in itself 
a further imposition both institutions could probably do without. Managing these conflicting 
demands is something that the Project Team needs to carefully monitor so as to not unduly 
distract the participating institutions from their main game, the HEPPP projects. So it will be 
very important for the Project Team to develop ways that allow the HEPPP projects to 
progress while still being able to monitor that progress in a non-invasive manner. 

Case Studies: Progress to date 

SCU’s position within Australia’s higher education system, as a relatively small and 
new university (established in 1994), needs to be considered in context. SCU has a diverse 
student population with school leavers representing less than 20% of commencing students. 
Around 60% of students are from a regional background, with 24% from a low SES 
background. Around one third study externally and almost 40% of students study part-time. 
SCU has been chosen because it currently exceeds the sector average for the participation 
rates of students from low SES, regional, rural, remote and Aboriginal and Torres Strait 
Islander backgrounds. 

Deakin University provides a good comparison to SCU. It has been working with the 
University Executive to introduce a student cohort tracking system since June 2010. The 
university has a diverse student body with regional and low SES participation proportions 
above the Victorian higher education average. One quarter of its students are admitted 
through the Special Entry Access Scheme, for people from disadvantaged backgrounds. 

As mentioned previously, this study is using an action research methodology within 
each university. Action Learning Teams (ALTs) are being led by Dr Jennifer Oriel, Manager, 
Student Access and Equity at Deakin University, and Ms Julie Arthur, Director Planning 
Quality and Review at Southern Cross University. ALTs comprise staff members from 
various backgrounds to ensure that all technical aspects of the project are considered. This 
includes teaching and learning academic expertise as well as data warehousing and analysis 
expertise. 

At Southern Cross University the Action Learning Team comprises the Director, 
Planning, Quality and Review (Chair), Director Equity, Head, Equity and Diversity, the 
Institutional Research Officer, MIS Corporate Systems Developer and Academic Directors of 
T&L. 


SCU has established an i-Ontrack Project being overseen by the i-Ontrack Project 
Reference Group chaired by the Deputy Vice-Chancellor. SCU’s i-Ontrack project has a 
number of key objectives: 

1. Measure the impact of teaching and learning activities to improve the participation, 
progress and completion of students from low SES, low income and academically 
underprepared backgrounds; 

2. Monitor outcomes related to student cohorts effectively at the institutional level in 
order to gain a better understanding of the root causes of disadvantage as well as 
tracking the enablers of access, retention and success of students from diverse 
backgrounds; 
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3. Develop a tracking system that follows a cohort of students who come from diverse 
backgrounds in order to identify those factors in their life that impede or boost 
academic excellence; 

4. Identify the most effective ways of extending the foundation of technologies and data 
sets available at SCU to build a platform for identifying, planning and monitoring of 
‘at risk’ students; and 

5. Enable the delivery of a retention and success planning capability which can: 

• Identify students ‘at risk’; 

• Plan key retention metrics and show performance against plans 

• Plan intervention methods to be used for an identified set of ‘at risk’ students 

• Identify data capabilities for an early warning system 

• Identify best point of intervention 

• Support an intervention model. 

With respect to point 5 above, SCU is partnering with a consultant in order to develop 
a predictive model for students most at risk. In order to systematically address these 
conditions, SCU is also redefining, or at least reconsidering, much of the terminology it uses 
in this regard. Terms such as attrition and at risk are being carefully reconsidered and the 
exclusion policy is in the process of being rewritten to more adequately account for at risk 
students. It has also been proposed that these changes to terminology and policy should apply 
to both domestic and international students. 

Peakin’s Diverse Student Cohort Tracking Committee (DSCTC) was established in 
July 2010 to prepare university data systems for new regulatory requirements arising from the 
Commonwealth’s policy paper ‘Transforming Australia’s Higher Education System'. In 
particular, the government’s focus on student participation indicators in the Higher Education 
Participation and Partnerships Program (HEPPP), and teaching and learning performance 
standards in the draft ‘Indicator Framework for Higher Education Performance Funding’, 
demonstrated the need for evidence-based policymaking at the local and national levels. 

Peakin’s success in attracting funding for a number of HEPPP projects provides a 
significant impetus for the introduction of cohort tracking under the program’s auspices and 
funding. 

In a briefing paper to the Committee, its founding Chair, Dr Jennifer Oriel established 
that its primary function was to: 

develop systems to track students from low SES backgrounds in the first instance, and to 
use this process to diagnose and address gaps in the existing student data systems towards 
consideration of a more mainstream cohort tracking model that could inform business 
intelligence capabilities. (Oriel, 2011c, p. 1) 

Oriel identified four basic requirements for the introduction of student cohort tracking 
and noted that significant progress had been made towards the introduction of low SES 
student cohort tracking at the university, including (1) the purchase of the geo-coding 
software to match student addresses to collection districts, which replaces the postcode 
reference value previously used by DEEWR; (2) funding has been provided by Student 
Access and Equity for the employment of a HEPPP project officer and (3) discussions have 
occurred surrounding the interoperability between the data warehouse and the Student 
Information System to (4) allow the generating of reports related to student cohort tracking. 
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In the briefing paper, Oriel also identified that the following attributes were defined 
for data capture to enable cohort tracking of low SES students against the principles, 
objectives and indicators of the HEPPP, and the proposed Indicator Framework for Higher 
Education Performance Funding: 

• SES by Collection District— • 

low, medium and high 

• school of origin • 


• Australian Tertiary Admissions • 

Rank (ATAR) 

• Commonwealth Scholarship or • 

Deakin equity scholarship 
recipient 

• ‘first in family’ to higher • 

education 

While the identification of variables for predicting at risk students is important, 
universities tend to grapple with the interpretation of macro-level cohort data at the individual 
level. In a separate paper to the Vice-Chancellor in June 2011 on national indicators for low 
socioeconomic status. Dr Oriel advised that ’while it is not necessary to introduce causative 
analysis for all impact assessments, it is important to improve individual level reporting in 
response to the HEPPP’ (Oriel, 2011b, p. 4). She noted that, ‘Further impetus for the 
introduction of individual measures to assess project outcomes was provided in the NCVER 
report ‘Measuring the Socioeconomic Status of Australian Youth’, and that ‘while the SEIFA 
index provides a reasonably accurate measure of SES higher education participation at the 
aggregate level, it is a poor reference measure for individual student performance (Oriel, 

201 lb, p. 4).’ 

In a review of Lim and Gemici’s paper. Oriel explained that: 

A new composite SES measure based on a factor analysis model used in the Australian 
Council for Education Research Longitudinal Studies of Australian Youth (LSAY) 
included 14 different variables. The use of composite measures for socio-economic status 
derives from literature on the taxonomy of educational disadvantage which organises 
classifications according to multiple variables (Singleton, 2010). However, there are 
potential problems for evaluation of university programs against national policy objectives 
if different measures are employed to assess outcomes. In addition, there is evidence to 
suggest that when assessing equity performance, different variables affect performance 
against each indicator. For example, a 2010 analysis on attrition in Australian universities 
by Gavin Moddie found that the three main factors affecting student attrition were tertiary 
entrance scores, mode of study and off-campus or on-campus attendance. (Oriel, 2011c, p. 

2 ) 

Oriel concluded that 

Some Australian universities are beginning to respond to the complexity of improving 
academic outcomes for students from diverse backgrounds by designing their own 
composite measures based on institutional student profiles. Such developments indicate 


Special Entry Assessment 
Scheme (SEAS) applicant 

Grade Point Average (GPA)/ 
Weighted Average Mark 
(WAM) 

attrition—more clearly defined 
as discontinuation rather than as 
course change 

Bachelor degree completion 
rate 

use of student support services 
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that a broader SES measure may be needed, particularly to assess the impact of teaching 
and learning programs on students’ progress in response to the HEPPP. (Oriel, 2011c, p, 2) 

Deakin has strong partnerships with 40 schools across Victoria as a part of the Peak in 
Engagement and Access Program, which Dr Oriel (who is also the Manager of Student 
Access and Equity), established in 2007. In 2011, Dr Oriel designed a new program, Access 
Express, with seven schools for the purpose of positioning the university favourably to win 
the first round of Commonwealth competitive funding under the HEPPP. Among 35 bids 
submitted from universities across Australia, Access Express was one of eleven selected by 
the Minister for funding. It introduces impact evaluation methodologies for university-school 
partnerships that are enabled by cohort tracking. 

SCU’s approach has drawn on international research, particularly in the area of 
academic or learning analytics. The i-Ontrack Committee have mostly focused on what data 
are currently available through SCU’s information systems that can be analysed to provide a 
picture of a students’ interaction with the university as a pseudo measure of their level of 
engagement with the university. 

A number of commentators (Fritz, 2011; Zhang Almeroth, Knight, Bulger, & Mayer, 
2010) discuss the relationship between the use of activity data of the university’s Course 
Management System (such as Blackboard®, Web CT™ or Moodle™) and student success. In 
both cases, the intention was to provide students an early indication of their own usage of 
these systems compared with the usage of the broader university population as they believed 
that showing students their relative level of engagement with the CMS prompted a 
motivational response. One such response may be to seek academic assistance at an earlier 
point of time in their studies (Fritz, 2011). 

SCU is therefore interested in obtaining as much data as possible from their CMS and 
from their other information systems. One other data source SCU has identified is the 
students’ usage of their libraries’ resource collections. Unsurprisingly, many of these systems 
do not provide, as a general rule, analytics of the kind necessary to monitor student activity— 
at least in a form that can be easily integrated with a monitoring system. Therefore, one of 
SCU’s early considerations will be to understand what analytics can be obtained currently 
and what could be obtained if additional resources were applied to the task. 

To assist the development of a predictive model for identifying students at risk, SCU 
have engaged the services of an external consultant with experience in the area of predictive 
modelling. The consultant is using the various data sources available as inputs into the model 
they are developing. 

SCU’s intention would be to develop intervention strategies appropriate and focused 
on those students who exhibit at risk tendencies, identified using the predictive modelling, 
early in the students’ tertiary studies. 

By contrast, Deakin University are focusing their data collection on those student 
characteristics that pertain mostly to their pre-tertiary circumstance (e.g., tertiary entrance 
scores, ‘first in family’ and socioeconomic status) in conjunction with study mode and 
academic performance in their tertiary studies. In this way Deakin are engaging with their 
participating schools in order to derive a fuller picture of the student’s pre-tertiary 
circumstances. They believe they may be able to focus their interventions on potential 
students while they are still within their pre-tertiary environments, namely schools. 
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Of course, the challenges here are quite different: SCU is trying to collect data from 
its own information systems over which it has complete control (although limited by the 
technology); Deakin is trying to understand the pre-tertiary environment, but having less 
control over being able to influence school policy and so forth. 

Teams at each institution are ‘learning by doing’ as they document their learning 
process, which, in the first stage of this project, will be the design of an effective student 
cohort tracking system. The two teams are collecting data about their own learning process 
and are also conducting research to inform their practice along the way. 

Conclusion 

The project has chosen two institutions to participate in an action research process 
that have adopted quite different approaches to the implementation of their HEPPP projects. 
On the one hand, Deakin University has a large number of HEEP projects it is managing 
through project teams coordinated through a centralised committee, the Diverse Student 
Cohort Tracking Committee. These teams are themselves determining what data needs to be 
collected, the best way to collect the cohort data and how best to evaluate the effectiveness of 
any interventions they are monitoring. 

Deakin is concerned with a number of HEPPP projects, mostly dealing with data 
pertaining to their students’ pre-tertiary attributes such as tertiary entrance score, ‘first in 
family’ and socioeconomic status. The university has partnered with a number of secondary 
colleges in their catchment in order to assist them in these projects. 

SCU, on the other hand, is focused on one HEPPP project dealing with the 
development of a predictive model that will be able to identify particular cohorts of students 
who may be at risk and the type of risks to which they may be susceptible. An external 
consultant is assisting them with the development of such a predictive model. The data they 
are collecting relates to student attributes during the early stage of the university study, such 
as LMS and library usage. 

Both universities are considering the data that are needed, how that data are to be 
recorded and stored, and what reporting capabilities will be needed to interrogate that data. 
They are also interested in who should be involved in determining these matters and how to 
monitor the contributions the various individuals make. What processes they choose to use to 
manage their programs is also something that each institution is very keen to develop and 
understand. 

This longitudinal study will hopefully provide some insights into the various ways in 
which student cohort tracking projects can develop, the good practices that can be learnt, as 
well as those hard lessons that should possibly be avoided. By investigating methodologies 
that more effectively track student cohorts, this study will provide longitudinal and empirical 
evidence for the effect of current strategies to address educational disadvantage and 
recommendations for how to more effectively track performance into the future. Results from 
this study will also enable a better understanding of where teaching and learning 
interventions could be best placed to assist underprepared students’ success and completion 
in higher education. It will identify which life course conditions (such as high levels of 
student mobility) may need to be tracked so that students at risk can more easily be identified 
early and assisted to achieve successful outcomes. 
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At this early stage of the research, it is clear that implementing a further level of 
monitoring to the programs adds another level of complexity that each institution could 
possibly do without. However, the later benefits of understanding what provides for a 
successful cohort tracking system will undoubtedly benefit both institutions involved, as well 
as providing other institutions who wish to undertake such programs with much clearer 
implementation guidelines than may presently be the case. 
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